A storage server is a computer that provides remote clients with shared storage access, typically to disk storage. In order to manage input/output (I/O) operations from the remote clients, the storage server may use a message-based storage protocol. For example, when a remote client would like to read or write to shared disk, the remote client sends a message over a network to the storage server. An agent process running on the storage server responds to the request. The agent may process the message, perform the requested disk storage I/O operations, and return a confirmation message to the remote client. Thus, the storage protocol is two-sided as it involves central processing units (CPUs) on both sides of the connection executing processes that communicate with each other to coordinate I/O operations.
In some cases, two-sided storage protocols may use Remote Direct Memory Access (RDMA) to transfer data between the two computers. As the term is used herein, RDMA is a technology that allows the network interface controller (NIC) of the storage server to transfer data “directly” to or from memory of a remote client, that is, transferring the data to or from the memory without involving the central processing unit(s) (CPUs) on the remote client. In an example approach, the remote client registers a target memory buffer and sends a description of the registered memory buffer to the storage server. The remote client then issues a read or write request to the storage server. If the request is a write request, the storage server performs an RDMA read to load data from the target memory buffer into the storage server's local memory. The storage server then causes a disk controller to write the target data to disk and, once the write is complete, generates and sends a write confirmation message to the remote client. In response to a read request, the storage server uses the disk controller to perform a block-level read from disk and loads the data into its local memory. The storage server then performs an RDMA write to place the data directly into an application memory buffer of the remote computer. After an RDMA operation completes, the remote client deregisters the target memory buffer from the RDMA network to prevent further RDMA accesses. Using RDMA increases data throughput, decreases the latency of data transfers, and reduces the load on the storage server and remote client's CPU during data transfers. Although the CPU's incur some overhead from memory registration and message processing, such latency is typically much lower than the latency of writing to disk and, therefore, does not significantly impact performance.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.